Conference system, video conference apparatus, and video image processing method

ABSTRACT

A video conference apparatus includes a relay unit that is configured to transmit video image data acquired at a site of a first conference system to a second terminal provided in a second conference system and transmit video image data of the second conference system to a first terminal provided in the first conference system. The relay unit is configured to transmit only a combined video image of video images at respective sites of the first conference system to the second terminal as first video image data, and transmit a video image sent from the second terminal to the first terminal as second video image data.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority ofJapanese Patent Application No. 2020-202152 filed on Dec. 4, 2020, thecontents of which are incorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

The present disclosure relates to a conference system, a videoconference apparatus, and a video image processing method.

A video conference system that establishes a communication path betweena plurality of sites and transfers a video image and audio to conduct aconference has been used. Patent Literature 1, JP-A-2009-141508discloses a video conference apparatus that realizes a video conferencefor performing nursing care at connecting multiple points. The videoconference apparatus disclosed in JP-A-2009-141508 receives a tableimage transmitted from another video conference apparatus, generates atable image obtained by removing an image displayed on a display devicefrom an image obtained by imaging a surface including the display deviceof the conference table using the received image using the receivedimage, and transmits the generated table image to another videoconference apparatus. With this configuration, it is possible to reducethe cost while solving a problem of video loop, and to realize aflexible disposition of the devices according to a situation of aconference room.

Patent Literature 1: JP-A-2009-141508

SUMMARY OF THE INVENTION

An object of the present disclosure is to provide a conference system, avideo conference apparatus, and a video image processing method thatsuppress occurrence of looping of a video image when transferring thevideo image by making a plurality of conference systems cooperate witheach other and allows an appropriate conference video image to be sharedbetween the plurality of conference systems.

According to an aspect of the present disclosure, there is provided aconference system including a first terminal that is configured totransmit and receive video image data acquired at a site of a firstconference system, a second terminal that is configured to transmit andreceive video image data acquired at a site of a second conferencesystem, and a relay unit that is configured to mutually transfer databetween the first terminal and the second terminal, transmit first videoimage data to the second terminal, and transmit second video image datato the first terminal, in which the relay unit is configured to transmitonly a combined video image of video images at respective sites of thefirst conference system to the second terminal as the first video imagedata and transmit a video image sent from the second terminal to thefirst terminal as the second video image data.

According to another aspect of the present disclosure, there is provideda video conference apparatus including a relay unit that is configuredto transmit video image data acquired at a site of a first conferencesystem to a second terminal provided in a second conference system andtransmits video image data of the second conference system to a firstterminal provided in a first conference system, in which the relay unitis configured to transmit only a combined video image of video images atrespective sites of the first conference system to the second terminalas first video image data and transmit a video image sent from thesecond terminal to the first terminal as second video image data.

According to still another aspect of the present disclosure, there isprovided a video image processing method in a video conference apparatusincluding a relay unit that transmits video image data acquired at asite of a first conference system to a second terminal provided in asecond conference system and transmits video image data of the secondconference system to the first terminal provided in the first conferencesystem, the video image processing method includes transmitting only acombined video image of video images at respective sites of the firstconference system to the second terminal as first video image data andtransmitting a video image sent from the second terminal to the firstterminal as second video image data.

According to the present disclosure, occurrence of looping of a videoimage is reduced when transferring the video image by making a pluralityof conference systems cooperate with each other, and an appropriateconference video image can be shared between the plurality of conferencesystems.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example of a schematicconfiguration of a conference system according to a first embodiment;

FIG. 2 is a block diagram illustrating a functional configuration of anapparatus in the conference system according to the first embodiment;

FIG. 3 is a diagram illustrating a specific example during operation inthe conference system according to the first embodiment;

FIG. 4 is a flowchart illustrating an example of an operation procedurein the conference system according to the first embodiment;

FIG. 5 is a diagram illustrating a first example of a display screen inthe conference system according to the first embodiment;

FIG. 6 is a diagram illustrating an example of a display screen in aconference system of a comparative example.

FIG. 7 is a diagram illustrating a second example of the display screenin the conference system according to the first embodiment;

FIG. 8A is a diagram illustrating a first example of video imagetransfer processing and a display screen during operation in theconference system according to the first embodiment;

FIG. 8B is another diagram illustrating the first example of the videoimage transfer processing and the display screen during operation in theconference system according to the first embodiment;

FIG. 9A is a diagram illustrating a second example of the video imagetransfer processing and the display screen during operation in theconference system according to the first embodiment;

FIG. 9B is another diagram illustrating the second example of the videoimage transfer processing and the display screen during operation in theconference system according to the first embodiment;

FIG. 10 is a diagram illustrating an example of video image transferprocessing and a display screen during operation a conference system ofa comparative example.

FIG. 11 is a block diagram illustrating an example of a schematicconfiguration of a conference system according to a second embodiment;

FIG. 12A is a block diagram illustrating a functional configuration of adevice in the conference system according to the second embodiment;

FIG. 12B is a block diagram illustrating a functional configuration of adevice in the conference system according to the second embodiment;

FIG. 13A is a diagram illustrating a first example during operation inthe conference system according to the second embodiment;

FIG. 13B is a diagram illustrating a second example during operation inthe conference system according to the second embodiment; and

FIG. 14 is a block diagram illustrating an example of a server computerfor the cloud in the conference system according to the secondembodiment.

DESCRIPTION OF EMBODIMENTS Background of Embodiments

In a video conference system in which a video image and audio aretransferred between a plurality of sites to conduct a conference, thereis a demand for making different conference systems cooperate with eachother so that a plurality of conference systems share a video image toconduct a conference. For example, a configuration is conceivable inwhich a video conference system for conducting a video conference usinga video conference terminal between a plurality of sites and a Webconference system for conducting a Web conference via the Internet usinga terminal such as a PC are made to cooperate with each other to allowsharing of a conference video image between the two systems. Aconfiguration is also conceivable in which a system is connected toanother company's system having different specifications to allowsharing of a video image of a video conference between a plurality ofsystems.

When the plurality of conference systems are made to cooperate with eachother as described above, usually, the video images of each other'sconference systems are transferred, and conference video images of thecooperating counterpart systems are combined and displayed in eachconference system. In this case, there is a problem that a so-calledvideo loop occurs in which the combined conference video is returned tothe counterpart's conference system, and the same video image isrepeatedly transferred and a video image like two opposite mirrors isdisplayed by repetitive returning of the combined conference video tothe counterpart's conference system.

Hereinafter, embodiments which specifically disclose the configurationsof the conference system, the video conference apparatus, and the videoimage processing method according to the present disclosure will bedescribed in detail with reference to the drawings as appropriate.However, more detailed description than necessary may be omitted. Forexample, detailed descriptions of already well-known matters andrepeated descriptions for substantially the same configuration may beomitted. This is to avoid the following description from becomingunnecessarily redundant and to facilitate understanding by those skilledin the art. The accompanying drawings and the following description areprovided to enable those skilled in the art to fully understand thepresent disclosure, and are not intended to limit the claimed subjectmatter.

In the following, a plurality of examples of a conference system inwhich occurrence of looping of a video image is suppressed bytransferring without including a video image of its own site at a relaypoint when transferring the video image by making a plurality ofconference systems cooperate with each other and an appropriateconference video image can be shared between the plurality of conferencesystems will be described.

Embodiments

FIG. 1 is a block diagram illustrating an example of a schematicconfiguration of a conference system according to this embodiment. Inthis embodiment, as a configuration example of a video conference systemin which a plurality of conference systems are made to cooperate witheach other, a configuration in a case where a first conference systemand a second conference system are made to cooperate with each other bybeing connected to each other via a relay device at a specific site isexemplified. In the following, it is assumed that a video conferencesystem is used as an example of the first conference system, and a Webconference system is used as an example of the second conference system.

A site A is provided with a video conference terminal A (10A)(hereinafter also referred to as a “video conference terminal 10A”) asan example of a first terminal that transmits and receives a video imageand audio by the video conference system. The video conference terminal10A is connected to a video conference terminal B (10B) (hereinafter,also referred to as a “video conference terminal 10B”) provided at asite B and a video conference terminal C (10C) (hereinafter, “videoconference terminal 10C”) provided at a site C via a communicationchannel 60. As the communication channel 60, for example, an IP networkconstructed in a public data communication network or a closed datacommunication network is used. In the illustrated example, although aconfiguration in which three video conference terminals are connected isillustrated, a configuration in which two video conference terminals areconnected to each other, a configuration in which four or more videoconference terminals are connected to each other may be adopted, and thenumber of terminals is not limited.

The video conference terminals 10A, 10B, and 10C can share the videoimages and audio at respective sites between the terminals of the videoconference system by transferring the video image and audio data(hereinafter, also referred to as “video image and audio data”) of theirown sites acquired by imaging at their own terminals to a terminal atanother site. For example, the video conference terminal 10A becomes amaster terminal, acquires and combines the video of the terminal at eachsite, and transfers the combined video image of a plurality of sites tothe video conference terminals 10B and 10C at other sites. Then, thevideo conference terminals 10A, 10B, and 10C display and reproduce theacquired video image and audio on a display unit (see FIG. 2). With thisconfiguration, in each of the video conference terminals 10A, 10B, and10C connected to the video conference system, the combined video imageof the plurality of sites is displayed on the display unit (see FIG. 2),and a conference participant can view and share the combined video ateach site and grasp a state of each other's sites.

Also, at a site a, which is the same site as the site A, a Webconference terminal a (30 a) (hereinafter also referred to as a “Webconference terminal 30 a”) as an example of a second terminal thattransmits and receives a video image and audio by the Web conferencesystem is provided. The video conference terminal 10A is connected to arelay device 20 as an example of a relay unit that connects and makesthe video conference system and the Web conference system cooperate witheach other, and the video conference terminal 10A and the Web conferenceterminal 30 a are connected via the relay device 20. The relay device 20is also called a gateway box (GWB). As an example of the videoconference apparatus, a video conference terminal 10 a which isconfigured as an integrated device including the video conferenceterminal 10A and the relay device 20 and has a function of the relaydevice 20 may be provided.

The Web conference terminal 30 a is connected via a network 50 to a Webconference terminal b (30 b) (hereinafter, also referred to as “Webconference terminal 30 b”) provided at a site b. As the network 50, forexample, an IP network constructed in a public data communicationnetwork such as the Internet is used. In the illustrated example,although a configuration in which two Web conference terminals areconnected is illustrated, a configuration in which three or more Webconference terminals are connected may be adopted, and the number ofterminals is not limited.

The Web conference terminals 30 a and 30 b can share the video imagesand audio at respective sites between the terminals of the Webconference system by transferring the video image and audio data oftheir own sites acquired by imaging at their own terminals to a terminalat another site. For example, in each of the Web conference terminals 30a and 30 b, the video image of its own terminal and the video image ofthe other terminal are combined, and the acquired video image and audioare displayed and reproduced on the display unit (see FIG. 2). With thisconfiguration, in each of the Web conference terminals 30 a and 30 bconnected to the Web conference system, the combined video image of theplurality of sites is displayed on the display unit (see FIG. 2), and aconference participant can view and share the combined video at eachsite and grasp states of each other's sites.

In a first embodiment, the relay device 20 transfers the video image andaudio data of the video conference system acquired by the videoconference terminal 10A as an example of first video image data to theWeb conference terminal 30 a. The relay device 20 transfers the videoimage and audio data of the Web conference system acquired by the Webconference terminal 30 a as an example of second video image data to thevideo conference terminal 10A. The video conference terminal 10Atransfers the video image and audio data of the Web conference systemtransferred from the Web conference terminal 30 a via the relay device20 to the video conference terminals 10B and 10C at other sites as anexample of a third terminal. The Web conference terminal 30 a transfersthe video image and audio data of the video conference systemtransferred from the video conference terminal 10A via the relay device20 to the Web conference terminal 30 b at another site as an example ofa fourth terminal. With this configuration, the video image and audiocan be mutually shared between a plurality of conference systems of thevideo conference system and the Web conference system. The number ofconference systems interconnected by the relay device 20 is not limitedto two in the illustrated example, a configuration in which three ormore conference systems are connected may be adopted, and the number ofthe first conference systems and/or the second conference systems is notlimited. In addition, information sharing between a plurality ofconference systems is not limited to the video image data and audio dataobtained by imaging a conference participant at each site, and variouscontents such as image data such as conference materials, moving imagedata, and sound data can be mutually transferred and shared with otherconference systems.

FIG. 2 is a block diagram illustrating a functional configuration of anapparatus in the conference system according to this embodiment. Thevideo conference terminal 10A includes a communication unit 11 and asignal processing unit 12, and is connected to a display unit 13, animaging unit 14, and a sound collection unit 15. The display unit 13includes a display device such as a liquid crystal display or an organicelectroluminescence (EL) display, and displays a video image in theconference system. The imaging unit 14 includes, for example, a cameraincluding an imaging lens and an imaging device such as an image sensor,and acquires video image data by imaging contents such as conferenceparticipants or materials at the site. The sound collection unit 15 isconfigured by a sound collection device such as a microphone, forexample, and collects sound at a site to acquire audio data. Thecommunication unit 11 includes a communication protocol such as asession initiation protocol (SIP), H.323, and the like, and communicateswith a terminal at another site of the video conference system and therelay device 20, respectively, to transmit and receive video image andaudio data and control data. The signal processing unit 12 includes aprocessor and a memory, and executes signal processing such as encodingand decoding of video image and audio data, video image combining, andvideo image division.

The relay device 20 includes a communication unit 21 and a signalprocessing unit 22. The communication unit 21 includes a communicationinterface, and communicates with the video conference terminal 10A andthe Web conference terminal 30 a, respectively, to transmit and receivevideo image and audio data and control data. The signal processing unit22 includes a processor and a memory, and executes signal processingsuch as encoding and decoding of video image and audio data, video imagecombining, and video image division. The video conference terminal 10Aand the relay device 20 are connected by a communication path such as alocal area network (LAN). The Web conference terminal 30 a and the relaydevice 20 are connected by a communication path corresponding to acommunication standard such as high-definition multimedia interface(HDMI) (registered trademark) and universal serial bus (USB). The Webconference terminal 30 a and the relay device 20 are connected by, forexample, an HDMI (registered trademark) cable and a USB cable, andbidirectional data transfer is performed between the two devices byallowing video image and audio data to be transferred using respectivecommunication cables. For example, with respect to the relay device 20,video image and audio data are transferred from the Web conferenceterminal 30 a to the relay device 20 by an HDMI (registered trademark)output of the terminal. In addition, the video image and audio data aretransferred from the relay device 20 to the Web conference terminal 30 aas inputs to the camera and the microphone by a USB input of theterminal.

The Web conference terminal 30 a is configured by an informationprocessing terminal such as a notebook PC or a tablet terminal, includesa communication unit 31, a signal processing unit 32, and is connectedto a display unit 33, an imaging unit 34, and a sound collection unit35. When the Web conference terminal 30 a is connected to the videoconference terminal 10A via the relay device 20, the Web conferenceterminal 30 a turns off the functions of the imaging unit 34 and thesound collection unit 35, and shares the imaging unit 14 and the soundcollection unit 15 of the video conference terminal 10A. Thecommunication unit 31 includes a communication interface correspondingto a communication system such as (Web Real-Time Communications (WebRTC) or Skype for Business, and communicates with a terminal at anothersite of the Web conference system and the relay device 20, respectively,to transmit and receive the video image and audio data and control data.The signal processing unit 32 includes a processor and a memory, andexecutes signal processing such as encoding and decoding of video imageand audio data, video image combining, and video image division.

FIG. 3 is a diagram illustrating a specific example during operation inthe conference system according to the first embodiment. The firstembodiment includes a cooperating unit CN that makes a video conferencesystem and a Web conference system cooperate with each other as aconceptual system configuration that makes a plurality of conferencesystems cooperate with each other. The cooperating unit CN connects thevideo conference system MS1 and the Web conference system MS2, andrelays the video image and audio data by relaying the two conferencesystems.

At the site A (site a) functioning as the cooperating unit CN, the videoimages A and a of the conference participants at the site A are imagedand acquired by the video conference terminal 10A. Here, the video imageA is a video image for the video conference system MS1, and the videoimage a is a video image for the Web conference system MS2. At the siteB of the video conference system MS1, the video image B of theconference participant at the site B is imaged, acquired by the videoconference terminal 10B, and transferred to the video conferenceterminal 10A at the site A. At the site C of the video conference systemMS1, the video image C of the conference participant at the site C isimaged and acquired by the video conference terminal 10C, andtransferred to the video conference terminal 10A at the site A.

The video conference terminal 10A at the site A performs a process ofcombining the video image A, the video image B, and the video image C atthe three sites, transmits the generated combined video image ABC to thevideo conference terminal 10B at the site B and the video conferenceterminal 10C at the site C, and displays the combined video image ABC onthe display unit (see FIG. 2) of its own terminal. The video conferenceterminal 10B at the site B and the video conference terminal 10C at thesite C each receive the combined video image ABC and display thecombined video image ABC on the display unit (see FIG. 2) of their ownterminal. With this configuration, in each of the sites A, B, and C, thecombined video image ABC is displayed on the display unit (see FIG. 2),and the video image of each site during the conference is shared.

The video conference terminal 10A outputs the video image A (video imagea) of its own site A or the combined video image ABC of the videoconference system to the relay device 20. The relay device 20 transmitsthe video image a or the combined video image ABC acquired from thevideo conference terminal 10A to the Web conference terminal 30 a. Onthe other hand, at the site b of the Web conference system MS2, thevideo image b of the conference participant at the site b is imaged,acquired by the Web conference terminal 30 b, and transferred to the Webconference terminal 30 a at the site a.

In the single Web conference system MS2, the Web conference terminal 30a transmits the video image a of its own site a to the Web conferenceterminal 30 b of the site b and receives the video image b of the site bfrom the Web conference terminal 30 b, and the video image a of its ownsite a and the video image b of another site b are combined anddisplayed on the display unit. The Web conference terminal 30 btransmits the video image b of its own site b to the Web conferenceterminal 30 a of the site a, receives the video image a of the site afrom the Web conference terminal 30 a, and combines the video image b ofits own site b and the video image a of another site a to be displayedon the display unit (see FIG. 2).

When the video conference system MS1 and the Web conference system MS2are connected and made to cooperate with each other by the relay device20, the relay device 20 transmits the combined video image ABC of thevideo conference system MS1 to the Web conference terminal 30 a. The Webconference terminal 30 a receives the combined video image ABCtransferred from the relay device 20 instead of the video image a of itsown site a, transmits the combined video image ABC to the Web conferenceterminal 30 b of the other site b, and combines the combined video imageABC and the video image b of another site b to be displayed on thedisplay unit (see FIG. 2). The Web conference terminal 30 b transmitsthe video image b of its own site b to the Web conference terminal 30 aof the other site a, and combines the video image b of its own site band the combined video image ABC transferred from another site a to bedisplayed on the display unit (see FIG. 2).

In the first embodiment, when transmitting a video image from the videoconference terminal 10A to the Web conference terminal 30 a of the Webconference system side, the relay device 20 transfers only the combinedvideo image ABC of the respective sites A, B, and C of the videoconference system as the first video image data. Then, in the Webconference system, the relay device 20 transfers the combined videoimage ABC from the Web conference terminal 30 a to the Web conferenceterminal 30 b at another site b. When transmitting a video image fromthe Web conference terminal 30 a to the video conference terminal 10A ofthe video conference system, the relay device 20 transfers the videoimage sent from the Web conference terminal 30 a as the second videoimage data. Then, the relay device 20 transfers the video image usingthe video conference terminal 10A to the video conference terminals 10Band 10C at the sites B and C. With this configuration, the occurrence ofa video loop phenomenon in which a video image like a mirror withrepeated nesting of the combined video image is displayed is preventedwithout transferring the same video image.

FIG. 4 is a flowchart illustrating an example of an operation procedurein the conference system according to this embodiment. Hereinafter, adescription will be made on an example of processing of video image andaudio data in the relay device 20 and the video conference terminal 10Awhen a cooperative operation between a plurality of conference systemsis performed.

The relay device 20 and the video conference terminal 10A startcooperation between the video conference system and the Web conferencesystem based on a manipulation instruction input by a user of aconference participant. For example, when a cooperation startmanipulation from the normal video conference mode to the Web conferencecooperation mode is performed on the video conference terminal 10A atthe site A, the relay device 20 and the video conference terminal 10Astarts a cooperative operation for making the conference systemcooperate. The manipulation instruction input such as start ofcooperation and display switching in the conference system is input bythe user of the conference participant at the video conference terminal10A or another terminal, for example.

The relay device 20 receives the video image a from the Web conferenceterminal 30 a (S1). Then, the relay device 20 transfers the receivedvideo image a to the video conference terminal 10A, and the videoconference terminal 10A transmits the video image a to the videoconference terminals 10B and 10C at respective sites (S2). The videoconference terminal 10A receives the video image from the videoconference terminals 10B and 10C at the respective sites and combinesthe received video images (S3). In this case, the video image A, thevideo image B, and the video image C of each of the sites A, B, and C ofthe video conference system are received, and the received video imagesare combined to generate a combined video image ABC. Then, the videoconference terminal 10A transfers the combined video image ABC to therelay device 20, and the relay device 20 transmits the combined videoimage ABC to the Web conference terminal 30 a (S4).

The relay device 20 and the video conference terminal 10A determinewhether or not a manipulation instruction input for display switchingfor the conference video image is input (S5). The relay device 20 andthe video conference terminal 10A repeatedly execute processing of S1 toS4 until a display switching manipulation is performed by the user ofthe conference participant and the instruction input for displayswitching is received. With this configuration, the video conferencesystem and the Web conference system can transmit and receive eachother's video images and can share the video images.

When the manipulation instruction input for display switching of theconference video image by the user of the conference participant isreceived (Yes in S5), the relay device 20 and the video conferenceterminal 10A switch the video image display mode. This display switchingoperation is executed when a display switching manipulation is performedso that only the video image b of the Web conference terminal 30 b isdisplayed on the Web conference terminal 30 a, and the display switchingmanipulation for disabling a video loop avoidance function describedlater is performed. In this case, the relay device 20 and the videoconference terminal 10A combine the video image A of the videoconference terminal 10A, the video images B and C received from thevideo conference terminals 10B and 10C at the respective sites, and thevideo image b received from the Web conference terminal 30 a, andgenerates a combined video image ABCb (S6). Then, the relay device 20and the video conference terminal 10A transmit the combined video imageABCb to the video conference terminals 10B and 10C and the Webconference terminal 30 a at the respective sites (S7).

The relay device 20 and the video conference terminal 10A determinewhether or not the manipulation instruction input for display switchingis performed (S8). The relay device 20 and the video conference terminal10A repeatedly execute processing of S6 and S7 until the displayswitching manipulation is performed by the user of the conferenceparticipant and the manipulation instruction input for display switchingis received. With this configuration, the relay device 20 and the videoconference terminal 10A receive and combine the video images of the Webconference system and the video conference system, and transmit thecombined video image to the respective conference systems to allow thecombined video image to be shared.

When the manipulation instruction input for the display switching isreceived by the user of the conference participant (Yes in S8), therelay device 20 and the video conference terminal 10A proceed to theprocessing of S1 to S4 and switch the video image display mode.

Next, the operation of the video conference system according to thisembodiment will be described using a specific example of a display videoimage when a cooperative operation of a plurality of conference systemsis performed.

FIG. 5 is a diagram illustrating a first example of a display screen inthe conference system according to this embodiment. In FIG. 5, a videoimage of the site A of the video conference system is indicated by HD-A,a video image of the site B of the video conference system is indicatedby HD-B, a video image of the site C of the video conference system isindicated by HD-C, and a video image of the site b of the Web conferencesystem is indicated by PC-b. The site a of the Web conference system andthe site A of the video conference system are the same, and the videoimage of its own terminal indicated by a face image is the same as theHD-A.

The first example of the display screen illustrated in FIG. 5 is anexample in which a combined video image of the video image of the Webconference system and the video image of the video conference system isnot transmitted to the Web conference system. In this first example,only the combined video image of the video conference system istransmitted to the Web conference system without combining the videoimage of the Web conference system and the video image of the videoconference system, and the combined video image is shared between aplurality of conference systems.

Prior to the start of the cooperative operation between the videoconference system and the Web conference system, at the sites a and b ofthe Web conference system, although not illustrated, a combined videoimage PC-a/b obtained by combining the video images PC-a and PC-b ofeach other is displayed. At the sites A, B, and C of the videoconference system, although not illustrated, a combined video imageHD-A/B/C obtained by combining the video images HD-A, HD-B, and HD-C isdisplayed. Immediately after the start of the cooperative operation, atthe sites A, B, and C of the video conference system, in addition to thecombined video image HD-A/B/C, the combined video image PC-a/b of theWeb conference system is combined and displayed. In this specification,a plurality of combined video images are indicated in a form describedside by side using “/” in such a way that a video image obtained bycombining the video image HD-A, the video image HD-B, and the videoimage HD-C is described as a “combined video image HD-A/B/C”, a videoimage obtained by combining the video image PC-a and the video imagePC-b is described as a “combined video image PC-a/b, and a video imageobtained by combining the video image PC-b and the video image HD-A isdescribed as a “combined video image PC-b/HD-A”.

Within a short time after the cooperative operation is started, thevideo loop avoidance function is enabled, and only the combined videoimage HD-A/B/C of the respective sites A, B, and C is transmitted fromthe video conference system to the Web conference system. With thisconfiguration, at the site a of the Web conference system, the videoimage PC-b/HD-A/B/C obtained by combining the video image PC-b and thecombined video image HD-A/B/C is displayed. Also, the video image PC-bHD-A/B/C is transmitted from the Web conference system to the videoconference system. With this configuration, the video imagePC-b/HD-A/B/C is displayed at the sites A, B, and C of the videoconference system, as in the Web conference system. In this case, avideo loop is avoided without the same video image being repeatedlytransferred between the video conference system and the Web conferencesystem.

FIG. 6 is a diagram illustrating an example of a display screen in aconference system of a comparative example. This comparative example isan example of a display screen when a combined video image of the videoimage of the Web conference system and the video image of the videoconference system is transferred to the Web conference system. In thiscomparative example, the video image of the Web conference system andthe video image of the video conference system are combined andtransmitted to the Web conference system, and the video image is sharedbetween the plurality of conference systems.

Prior to the start of the cooperative operation between the videoconference system and the Web conference system, similarly as in thefirst example described above, the video image PC-a/b obtained bycombining the video images PC-a and the PC-b is displayed at the sites aand b of the Web conference system, and the video image HD-A/B/Cobtained by combining the video images HD-A, HD-B, and HD-C is displayedat the sites A, B, and C of the video conference system. Immediatelyafter the start of the cooperative operation, at the sites A, B, and Cof the video conference system, in addition to the combined video imageHD-A/B/C, the combined video image PC-a/b of the Web conference systemis combined and displayed.

Thereafter, the combined video image HD-A/B/C/PC-a/b is transferred fromthe video conference system to the Web conference system, and thecombined video image is transferred to the site b as the video image ofthe site a. For that reason, at the site a of the Web conference system,a video image obtained by combining the video image PC-b and thecombined video image HD-A/B/C/PC-a/b is displayed. Furthermore, thecombined video image PC-b/HD-A/B/C/PC-a b is transferred from the Webconference system to the video conference system, and as illustrated inthe upper part of FIG. 6, the combined video imageHD-A/B/C/PC-b/HD-A/B/C/PC-a/b is displayed, and by repeating suchtransfer of the combined video image, as illustrated in the lower partof FIG. 6, a video image like a mirror in which nesting of a combinedvideo image is repeated is displayed. By repeating such transfer of thecombined video image between the video conference system and the Webconference system, a video loop phenomenon occurs.

FIG. 7 is a diagram illustrating a second example of a display screen inthe conference system according to this embodiment. The second exampleof the display screen illustrated in FIG. 7 is an example when only thevideo image b of the Web conference terminal 30 b is set to be displayedon the Web conference terminal 30 a, and is an example of the displayscreen corresponding to the operations of S6 to S7 in FIG. 4. In thesecond example, only the video image b of the Web conference terminal 30b is displayed on the Web conference terminal 30 a provided with therelay device 20, and the video image is shared between a plurality ofvideo conference systems.

When the cooperative operation of the video conference system and theWeb conference system is started, the video images of the sites A, B,and C of the video conference system and the video image of the site bof the Web conference system are combined and the combined video imageHD-A/B/C/PC-b is displayed at the site A. Then, the combined video imageHD-A/B/C/PC-b is transferred to the sites B and C and the site b, anddisplayed at each site. In this case, the combined video image HD-A/B/Cis not displayed on the display unit (see FIG. 2) at the Web conferenceterminal 30 a, the combined video image HD-A/B/C is not combined withthe video image b by the Web conference terminal 30 a, and the samevideo image is not repeatedly transferred, and thus a video loop isavoided. In a state where the video image of its own site A is notdisplayed, when a manipulation instruction for display switching isreceived by a manipulation of a user of the conference participant orthe like, the operation may be switched from the operation of the firstexample described above to the operation of transmitting the combinedvideo image as in the second example. The operation may be switched soas to become a state where the video image of its own site A is notdisplayed as in the second example by receiving a manipulationinstruction of display switching by the manipulation of the user of theconference participant or the like.

Next, transition of a video image data transferred between sites and adisplay video image displayed at each site when a cooperative operationof a plurality of conference systems is performed will be describedusing a specific example.

FIGS. 8A and 8B are diagrams illustrating a first example of video imagetransfer processing and a display screen during operation in theconference system according to this embodiment.

The first example of the video image transfer processing and displayscreen illustrated in FIGS. 8A and 8B illustrates a sequence in a casewhere, at the site A where a relay device is provided, only a combinedvideo image of one video conference system is transmitted to anothervideo conference system without combining a video image of its own siteA. This first example corresponds to the operations of S1 to S4 in FIG.4 and the display screen of FIG. 5.

During a single video conference of the video conference systems betweenthe sites A, B, and C, the video conference terminals 10B and 10Ctransfer the video images B and C of the sites B and C, respectively,and a combined video image of video images A, B, and C is generated atthe video conference terminal 10A (T1). The video conference terminal10A transfers the combined video image HD-A/B/C to the video conferenceterminals 10B and 10C (T2). With this configuration, the combined videoimage HD-A/B/C is displayed on the display unit of each of the videoconference terminals 10A, 10B, and 10C.

During a single Web conference of the Web conference systems between thesites a and b, the Web conference terminal 30 b transfers the videoimage b of the site b (T3), and the Web conference terminal 30 atransfers the video image a of the site a (T4). Then, in each of the Webconference terminals 30 a and 30 b, the video image of another site iscombined, and the combined video image PC-a/b is displayed on thedisplay unit.

Here, when a manipulation instruction for conference cooperation isissued by the user of the conference participant, the relay device 20connects the video conference terminal 10A of the site A and the Webconference terminal 30 a of the site a, and relays a video image andaudio data between the video conference system and the Web conferencesystem (T5). With this configuration, the cooperative operation betweenthe two conference systems is started, and the video image and audiodata are transferred to each other. In the connection state of theconference system, the Web conference terminal 30 a transfers thecombined video image PC-a/b of the sites a and b of Web conference tothe relay device 20 and the video conference terminal 10A (T6). Thevideo conference terminals 10B and 10C transfer the video images B and Cof the sites B and C of the video conference to the relay device 20 andthe video conference terminal 10A (T7).

Immediately after the connection between the video conference system andthe Web conference system is before the video loop avoidance function isenabled. The relay device 20 and the video conference terminal 10Atransfer a combined video image obtained by combining the combined videoimage PC-a/b and the combined video image HD-A/B/C to the videoconference terminals 10B and 10C and the Web conference terminal 30 a.(T8). In this case, the combined video image PC-a/b and the combinedvideo image HD-A/B/C are transferred in each of the video conferencesystem and the Web conference system, and the combined video image ofthe two conference systems is displayed on the display unit of eachterminal (T9 to T14).

Immediately after the connection between the video conference system andthe Web conference system, the video loop avoidance function is enabled.The relay device 20 and the video conference terminal 10A transfer onlythe combined video image HD-A/B/C of the video conference system to theWeb conference terminal 30 a, and transfer the video image sent from theWeb conference system side to the counterpart terminal of the videoconference system (T15). In this case, the Web conference terminal 30 acombines the video image PC-b of the Web conference terminal 30 b withthe combined video image HD-A/B/C of the video conference system,displays the combined video image on the display unit, and transfers thecombined video image to the relay device 20 and the video conferenceterminal 10A. For that reason, the video image obtained by combining thevideo image PC-b and the combined video image HD-A/B/C is transferredfrom the relay device 20 and the video conference terminal 10A to thevideo conference terminals 10B and 10C, and is displayed on the displayunit. In this state, the combined video image HD-A/B/C of the videoconference and the video image PC-b of the Web conference aretransferred in each of the video conference system and the Webconference system (T16 to T21). As described above, the switching timeTP1 until the video loop avoidance function is enabled is executed in ashort time, for example, within 1 second. Then, in each of the videoconference system and the Web conference system, the transfer of thecombined video image HD-A/B/C and the video image PC-b is continuouslyperformed, and the combined video image in a state where the video imageof each site does not overlap is displayed on the display unit of eachterminal (T22 to T25).

In the connection state between the video conference system and the Webconference system, the operation described above is repeated, and thecombined video image HD-A/B/C and the video image PC-b are transferredbetween the conference systems (T26 to T33). With this configuration,the same video image is not repeatedly transferred, and the combinedvideo image is generated in a state where the video image of each sitedoes not overlap and is displayed on the display unit of each of the Webconference terminals 30 a and 30 b and the video conference terminals10A, 10B and 10C, and thus the video loop is avoided.

FIGS. 9A and 9B are diagrams illustrating a second example of the videoimage transfer processing and the display screen during operation in theconference system according to this embodiment.

The second example of the video image transfer processing and thedisplay screen illustrated in FIGS. 9A and 9B is a sequence when aninstruction to disable the video loop avoidance function is issued by amanipulation of the user or the like in a state where the video image ofits own site A is not displayed at the site A where the relay device isprovided. This second example corresponds to the operations of S6 to S7in FIG. 4 and the display screen of FIG. 7.

Operations (T51 to T54) during a single conference in each of the videoconference system and the Web conference system and operations (T55 toT57) when a manipulation instruction for conference coordination isissued by the user of the conference participant are the same as theoperations T1 to T7 of the first example illustrated in FIGS. 8A and 8B.In this case, the Web conference terminal 30 a is in a state where thevideo image A (video image a of the site a) of its own site A is notdisplayed, displays only the video image PC-b of the counterpartterminal is displayed on the display unit, and transfers the video imagePC-b to the relay device 20 and the video conference terminal 10A (T56).

Immediately after the connection between the video conference system andthe Web conference system is before the video loop avoidance function isenabled. The relay device 20 and the video conference terminal 10Atransfer the combined video image obtained by combining the video imagePC-b and the combined video image HD-A/B/C to the video conferenceterminals 10B and 10C and the Web conference terminal 30 a (T58). Inthis case, the video image PC-b and the combined video image HD-A/B/Care transferred in each of the video conference system and the Webconference system, and the combined video image of the two conferencesystems is displayed on the display unit of each terminal (T59-T64).

Immediately after the connection between the video conference system andthe Web conference system, the video loop avoidance function is enabled.The relay device 20 and the video conference terminal 10A transfer onlythe combined video image HD-A/B/C of the video conference system to theWeb conference terminal 30 a, and transfer the video image sent from theWeb conference system side to the counterpart terminal of the videoconference system (T65). In this case, the Web conference terminal 30 adisplays only the video image PC-b of the counterpart terminal on thedisplay unit, and transfers the video image PC-b to the relay device 20and the video conference terminal 10A. For that reason, the video imagePC-b of the site b of the Web conference is transferred from the relaydevice 20 and the video conference terminal 10A to the video conferenceterminals 10B and 10C, and displayed on the display unit. In this state,the combined video image HD-A/B/C of the video conference and the videoimage PC-b of the Web conference are transferred in each of the videoconference system and the Web conference system (T66 to T71). Asdescribed above, the switching time TP2 until the video loop avoidancefunction is enabled is executed in a short time, for example, within 1second.

Then, the transfer of the combined video image HD-A/B/C and the videoimage PC-b is continuously performed in each of the video conferencesystem and the Web conference system (T72 to T76). In the secondexample, the relay device 20 and the video conference terminal 10Atransfer the video image PC-b of video conference to the videoconference terminals 10B and 10C, and only the video image PC-b of theWeb conference is displayed on the display unit of each terminal of thevideo conference system.

In this state, for example, when an instruction to disable the videoloop avoidance function is issued by the manipulation of the user of theconference participant and the manipulation instruction for displayswitching for the conference video image is received (T77), the relaydevice 20 and the video conference terminal 10A disable the video loopavoidance function. In this case, the relay device 20 and the videoconference terminal 10A transfer the combined video image obtained bycombining the video image PC-b and the combined video image HD-A/B/C tothe video conference terminals 10B and 10C and the Web conferenceterminal 30 a in the same manner as before the video loop avoidancefunction is enabled (T81). In this case, the combined video imageHD-A/B/C of the video conference and the video image PC-b of the Webconference are transferred in each of the video conference system andthe Web conference system, and the combined video image of the twoconference systems is displayed on the display unit of each terminal(T78 to T85). In the second example, even in a state where the videoloop avoidance function is disabled, the same video image at the site Ais not repeatedly transferred. Accordingly, the combined video image isgenerated in a state where the video of each site does not overlap andis displayed on the display unit of each of the Web conference terminals30 a and 30 b and the video conference terminals 10A, 10B and 10C, andthus the occurrence of the video loop phenomenon is prevented.

FIG. 10 is a diagram illustrating an example of video image transferprocessing and a display screen during operation in the conferencesystem of the comparative example.

The comparative example illustrated in FIG. 10 illustrates a sequence ina case where, at the site A where the relay device is provided, thevideo image of its own site A and the video image of another site arecombined and displayed on the display unit and the combined video imageis transferred to the other site, and corresponds to the display screenof FIG. 6.

Operations (T101 to T104) during a single conference in each of thevideo conference system and the Web conference system and operations(T105 to T107) when a manipulation instruction for conferencecoordination is issued by the user of the conference participant are thesame as the operations T1 to T7 of the first example illustrated inFIGS. 8A and 8B.

The relay device 20 and the video conference terminal 10A transfer acombined video image obtained by combining the combined video imagePC-a/b and the combined video image HD-A/B/C to the video conferenceterminals 10B and 10C and the Web conference terminal 30 a (T108). Inthis case, the combined video image PC-a/b and the combined video imageHD-A/B/C are transferred in each of the video conference system and theWeb conference system, and the combined video image of the twoconference systems is displayed on the display unit of each terminal(T109 to T114).

In the connection state between the video conference system and the Webconference system, the operation described above is repeated, and thetransfer of the combined video image HD-A/B/C and the video image PC-a/bcontinues in each of the video conference system and the Web conferencesystem (T115 to T119). In the case of the comparative example, theprocess of combining and transferring the combined video image HD-A/B/Cand the video image PC-a/b is repeated, the same video image isrepeatedly transferred to cause a phenomenon of a video loop, and avideo image like a mirror with repeated nesting of the combined videoimage is displayed.

As described above, in this embodiment, the relay device 20 transmitsonly the combined video image of the video images at the respectivesites A, B, and C of the video conference system to the Web conferenceterminal 30 a, and transmits the video image sent from the another siteb of the Web conference system that does not include the sites A and awhere the relay device 20 is provided to the video conference terminal10A. With this configuration, it is possible to suppress that the samevideo image at their own sites A and a is repeatedly transferred betweena plurality of conference systems. For that reason, a video loop can beavoided, an appropriate video image display screen can be obtained ateach site, and a smooth video conference can be executed by sharing aconference video image between the sites. Accordingly, it is possible toimprove a display mode when sharing a conference video by making aplurality of video conference systems cooperate with each other.

As described above, the conference system of this embodiment includesthe video conference system MS1 as an example of the first conferencesystem and the Web conference system MS2 as an example of the secondconference system. The conference system of this embodiment includes thevideo conference terminal 10A as a first terminal for transmitting andreceiving video image data acquired at the site of the video conferencesystem MS1 and the Web conference terminal 30 a as a second terminal fortransmitting and receiving video image data acquired at the site of theWeb conference system MS2. The conference system of this embodimentincludes the relay device 20 as a relay unit that mutually transfersdata between the video conference terminal 10A and the Web conferenceterminal 30 a, transmits the first video image data to the Webconference terminal 30 a, and transmits the second video image data tothe video conference terminal 10A. The relay device 20 transmits onlythe combined video image of the video image at the respective sites ofthe video conference system to the Web conference terminal 30 a as thefirst video image data, and transmits the video image sent from the Webconference terminal 30 a to the video conference terminal 10A as thesecond video image data. With this configuration, a video loop isavoided without the same video image being transferred repeatedlybetween a plurality of conference systems.

In this embodiment, the video conference terminal 10A transmits acombined video image obtained by combining video images at a pluralityof sites of the video conference system MS1 to the video conferenceterminals 10B and 10C as third terminals provided at the respectivesites of the video conference system, and transmits the second videoimage data sent from the relay device 20 to the video conferenceterminals 10B and 10C at the respective sites of the video conferencesystem MS1 when transferring data to the Web conference terminal 30 a bythe relay device 20. With this configuration, the same video image isnot repeatedly transferred, and the combined video image is generated ina state where the video image of each site does not overlap and isdisplayed on each terminal. For that reason, it is possible to preventthe occurrence of the video loop phenomenon in which the video imagelike a mirror with repeated nesting of the combined video image isdisplayed.

In the first embodiment, the Web conference terminal 30 a transmits thevideo image at its own site of the Web conference system MS2 to the Webconference terminal 30 b as a fourth terminal provided at another siteof the Web conference system, and combines the video image of the firstvideo image data sent from the relay device 20 with the video image atits own site, displays the combined video image on the display unit, andtransmits the video image of the first video image data to the Webconference terminal 30 b at another site of the Web conference systemMS2, when transferring data to the video conference terminal 10A by therelay device 20. With this configuration, the same video image is notrepeatedly transferred, and the combined video image is generated in astate where the video image of each site does not overlap and isdisplayed on each terminal. For that reason, it is possible to preventthe occurrence of the video loop phenomenon in which the video imagelike a mirror with repeated nesting of the combined video image isdisplayed.

The video conference apparatus according to this embodiment includes therelay device 20 that transmits the video image data acquired at the siteof the first conference system (video conference system MS1) to the Webconference terminal 30 a as a second terminal provided in the secondconference system (Web conference system MS2), and transmits the videoimage data of the second conference system to the video conferenceterminal 10A as the first terminal provided in the first conferencesystem. The video conference apparatus is configured as, for example, avideo conference terminal 10 a including the functions of the videoconference terminal 10A and the relay device 20. The relay device 20transmits only the combined video image of the video images at therespective sites of the video conference system MS1 to the Webconference terminal 30 a as the first video image data, and transmitsthe video image sent from the Web conference terminal 30 a as the secondvideo image data to the video conference terminal 10A. With thisconfiguration, a video loop is avoided without the same video imagebeing transferred repeatedly between a plurality of conference systems.

Second Embodiment

FIG. 11 is a block diagram illustrating an example of a schematicconfiguration of a conference system according to a second embodiment.Since the description of the same or equivalent portion as that of thefirst embodiment described above is duplicated, the description may beomitted or simplified by adding the same or equivalent referencenumerals to the drawings.

In the above-described example of the first embodiment, each function ofthe video conference terminal as an example of the first terminal, therelay device as an example of the relay unit, and the Web conferenceterminal as an example of the second terminal is provided by a physicalconfiguration mechanically realized by hardware. On the other hand, inthe second embodiment, the functions of these devices are realized bysoftware such as a program in a management terminal device CL (describedlater) as a server computer, and a configuration in that case isexemplified.

As illustrated in FIG. 11, in the second embodiment, a site A is notprovided as a physical site, but is a virtual site (hereinafter, alsoreferred to as the “virtual site A”) that is logically or virtuallyprovided on software (for example, Internet space). The same applies tosites a1 and a2, which are the same sites as site A (hereinafter, eachis also referred to as the “virtual site a1” or the “virtual site a2”).In the second embodiment, one management terminal device CL is logicallyinstalled across the virtual sites A, a1, and a2. In other words, themanagement terminal device CL is configured to logically include thesevirtual sites A, a1 and a2.

The management terminal device CL is configured of a dedicated devicefor a so-called server computer for the cloud in terms of hardware. Aprogram that is stored and held as software in a storage unit such asthe read only memory (ROM) or random access memory (RAM) of thespecialized computer is executed by a signal processing unit (describedlater) such as the central processing unit (CPU) to realize variousfunctions. The management terminal device CL is not limited to thededicated server computer described above, and may be configured by, forexample, a general-purpose computer such as a desktop computer or alaptop computer. Further, in the second embodiment, the managementterminal device is configured of a single computer, but is not limitedto thereto, and may be configured of a plurality of computers. FIG. 14is a block diagram illustrating an example of the server computer forthe cloud in the conference system according to the second embodiment.The server computer includes a central processing unit (CPU) CL1, a readonly memory (ROM) CL2, a random access memory (RAM) CL3, a displaydevice CL4, an input device CL5, a storage device CL6, and a networkinterface CL7 which are connected to a bus CL8. The central processingunit CL1 reads and executes a software program that realizes eachfunction from the read only memory CL2. Values and the like generated inarithmetic processing are temporarily written in the random accessmemory CL3. The display device CL4 is, for example, a liquid crystalmonitor, and displays the result of processing executed by the servercomputer. As the input device CL5, for example, a keyboard, a mouse, orthe like is used, and a predetermined operation input can be performed.As the storage device CL6, for example, a hard disk drive, a solid statedrive, an optical disk, a non-volatile memory, or the like is used. Aprogram allowing an operating system and a server computer to functionis also recorded in the storage device CL6. As the network interfaceCL7, for example, a network interface card or the like is used, andvarious data can be transmitted and received between devices via a localarea network, a wide area network, and the like. For example, thecentral processing unit CL1, the read only memory CL2, the random accessmemory CL3, and the storage device CL6 realize functions of signalprocessing units 112, 122 a 1, and 132 a 1 which are described later,and the network interface CL7 realizes functions of communication units111 and 131 a 1 which are described later.

The management terminal device CL has, as a function realized bysoftware, a video conference multipoint connection unit A (110A)(hereinafter, also referred to as the “video conference multipointconnection unit 110A”) as an example of a first transmission/receptionunit, a relay unit a1 (120 a 1) (hereinafter, also referred to as the“relay unit 120 a 1”) and a relay unit a2 (120 a 2) (hereinafter, alsoreferred to as the “relay unit 120 a 2”) as an example of a relay unit,a Web conference connection unit a1 (130 a 1) (hereinafter, alsoreferred to as the “Web conference connection unit 130 a 1”) and a Webconference connection unit a2 (130 a 2) (hereinafter, also referred toas “Web conference connection unit 130 a 2”) as an example of a secondtransmission/reception unit. The Web conference connection unit a1 islogically provided at the virtual site a1 described above, and the Webconference connection unit a2 is provided at the virtual site a2described above.

The video conference multipoint connection unit 110A of the managementterminal device CL is installed at the virtual site A, and is connectedto each of a video conference terminal 110B and a video conferenceterminal 110C as an example of a third transmission/reception unit via acommunication line 160. The video conference terminal 110B is installedat the physically provided site B according to the first embodiment, andthe video conference terminal 110C is also installed at the physicallyprovided site C according to the first embodiment. The video conferencemultipoint connection unit 110A of the management terminal device CLacquires and combines the video of the terminal at each site as amanagement terminal, and transfers the combined video image of aplurality of sites to the video conference terminals 110B and 110C atother sites. That is, in the second embodiment, a video conferencesystem MS11 as an example of the first conference system is configuredto include the video conference multipoint connection unit 110A, thevideo conference terminal 110B, the video conference terminal 110C, andthe communication line 160.

The relay unit 120 a 1 of the management terminal device CL is connectedto each of the video conference multipoint connection unit 110A and theWeb conference connection unit 130 a 1 of the management terminal deviceCL by software. Similarly, the relay unit 120 a 2 of the managementterminal device CL is also connected to each of the video conferencemultipoint connection unit 110A and the Web conference connection unit130 a 2 by software. The video conference multipoint connection unit110A of the management terminal device CL is connected to the Webconference connection units 130 a 1 and 130 a 2 of the managementterminal device CL via the relay units 120 a 1 and 120 a 2 of themanagement terminal device CL, respectively, that is, the relay units120 a 1 and 120 a 2 of the management terminal device CL connect andcooperate with the video conference system MS11 and the Web conferencesystem MS12.

Further, in the second embodiment, a plurality (two in the secondembodiment) of networks 150 are provided, and the Web conferenceterminal b1 (130 b 1) (hereinafter, also referred to as the “Webconference terminal 130 b 1”) of the management terminal device CL isphysically installed at the site b1 on one side of the network 150. TheWeb conference connection unit 130 a 1 of the management terminal deviceCL is connected to the Web conference terminal 130 b 1 as an example ofthe fourth transmission/reception unit via one of the networks 150. AWeb conference terminal b2 (130 b 2) (hereinafter, also referred to as“Web conference terminal 130 b 2”) is physically installed at the siteb2 on the other side of the network 150. The Web conference connectionunit 130 a 2 of the management terminal device CL is connected to theWeb conference terminal 130 b 2 as an example of a fourthtransmission/reception unit via the other network 150.

The Web conference connection units 130 a 1 and 130 a 2 of themanagement terminal device CL transfer the video image and audio data ofthe other sites to the terminals of the other sites, thereby makingvideo image and audio shareable at each site between the terminals inthe network 150 of the Web conference system MS12. That is, in thesecond embodiment, the Web conference system MS12 as an example of thesecond conference system is provided differently from the videoconference system MS11 as an example of the first conference system, andis configured to include the Web conference terminals 130 b 1 and 130 b2, and network 150. Further, the network 150 uses an IP networkconstructed in, for example, a public data communication network such asthe Internet, and is provided differently from a communication protocolof the video conference system MS12.

In the second embodiment, the relay units 120 a 1 and 120 a 2 of themanagement terminal device CL transfers the video image and audio dataof the video conference system MS12 acquired by the video conferencemultipoint connection unit 110A as an example of the first video imagedata to the Web conference connection units 130 a 1 and 130 a 2 of themanagement terminal device CL. Further, each of the relay units 120 a 1and 120 a 2 of the management terminal device CL transfers the videoimage and audio data of the Web conference system MS12 acquired by theWeb conference connection units 130 a 1 and 130 a 2 as an example of thesecond video image data to the video conference multipoint connectionunit 110A of the management terminal device CL. The video conferencemultipoint connection unit 110A of the management terminal device CLtransfers the video image and audio data of the Web conference systemtransferred from each of the Web conference connection units 130 a 1 and130 a 2 of the management terminal device CL via each of the relay units120 a 1 and 120 a 2 of the management terminal device CL to the videoconference terminal 110B and the video conference terminal 110C ofanother site as an example of the third terminal. Each of the Webconference connection units 130 a 1 and 130 a 2 of the managementterminal device CL transfers the video image and audio data of the videoconference system MS11 transferred from the video conference multipointconnection unit 110A via each of the relay units 120 a 1 and 120 a 2 ofthe management terminal device CL to each of the Web conferenceterminals 130 b 1 and 130 b 2 of another site as an example of thefourth terminal. With this configuration, the video image and audio canbe mutually shared between a plurality of conference systems of thevideo conference system MS11 and the Web conference system MS12. Thatis, in the second embodiment, the management terminal device CLconceptually functions as a cooperating unit CN referred to in the firstembodiment.

FIGS. 12A and 12B are block diagrams illustrating a functionalconfiguration of a device in the conference system according to thesecond embodiment.

As illustrated in FIGS. 12A and 12B, the video conference multipointconnection unit 110A of the management terminal device CL includes acommunication unit 111 and a signal processing unit 112. Thecommunication unit 111 of the video conference multipoint connectionunit 110A includes a communication interface and communicates withterminals at other sites of the video conference system MS11 and each ofthe relay units 120 a 1 and 120 a 2 of the management terminal deviceCL, and transmits and receives the video image and audio data, as wellas control data. The signal processing unit 112 of the video conferencemultipoint connection unit 110A executes signal processing such asencoding and decoding of video image and audio data, video imagecombining, and video image division.

Each of the relay units 120 a 1 and 120 a 2 has communication units 121a 1 and 121 a 2, and signal processing units 122 a 1 and 122 a 2. Eachof the communication units 121 a 1 and 121 a 2 of the relay units 120 a1 and 120 a 2 includes a communication interface, and communicates witheach of the video conference multipoint connection unit 110A and the Webconference connection units 130 a 1 and 130 a 2 of the managementterminal device CL, and transmits and receives the video image and audiodata, and control data. Each of the signal processing units 122 a 1 and122 a 2 of the relay units 120 a 1 and 120 a 2 executes signalprocessing such as encoding and decoding of video image and audio data,video image combining, and video image division.

Each of the Web conference connection units 130 a 1 and 130 a 2 includesa communication unit 131 and a signal processing unit 132. Each of thecommunication units 131 a 1 and 131 a 2 of the Web conference connectionunits 130 a 1 and 130 a 2 includes a communication interface, andcommunicates with the terminals of other sites of the Web conferencesystem and each of the relay units 120 a 1 and 120 a 2, and transmitsand receives the video image and audio data, and the control data. Eachof the signal processing units 132 a 1 and 132 a 2 of the Web conferenceconnection units 130 a 1 and 130 a 2 executes signal processing such asencoding and decoding of video image and audio data, video imagecombining, and video image division.

FIG. 13A is a diagram illustrating a first example during operation inthe conference system according to the second embodiment. FIG. 13B is adiagram illustrating a second example during operation in the conferencesystem according to the second embodiment.

In the second embodiment, the function (see FIG. 3) of the cooperatingunit CN referred to in the first embodiment is realized by themanagement terminal device CL. The management terminal device CL isprovided, for example, on the cloud and as a system configuration forcooperating a plurality of conference systems, connects the videoconference system MS11 and the Web conference system MS12, and relaysthe video image and audio data by relaying the two conference systems.

At the site B of the video conference system MS11, the video image B ofthe conference participant at the site B is imaged, acquired in thevideo conference terminal 110B, and transferred to the video conferencemultipoint connection unit 110A of the management terminal device at thevirtual site A. At the site C of the video conference system MS11, thevideo image C of the conference participant at the site C is imaged andacquired in the video conference terminal 110C, and transferred to thevideo conference multipoint connection unit 110A of the managementterminal device at the virtual site A. As described above, the virtualsite A and the sites a1 and a2 are logically provided sites, and novideo image is generated because imaging is not performed. In the secondembodiment, the virtual sites A, a1, and a2 function as cooperatingpoints as cooperating units of the management terminal device CL.

The video conference multipoint connection unit 110A of the managementterminal device CL performs a process of combining the video image B andthe video image C of the two sites, and transmits the generated combinedvideo image BC to the video conference terminal 110B of the site B andthe video conference terminal 110C of the site C. The video conferenceterminal 110B at the site B and the video conference terminal 110C atthe site C each receive the combined video image BC and display thecombined video image BC on the display unit of their own terminal. Withthis configuration, in each of the sites B and C, each combined videoimage BC is displayed on the display unit, and the video image of eachsite during the conference is shared.

Further, the video conference multipoint connection unit 110A of themanagement terminal device CL outputs the combined video image BC of thevideo conference system to the relay units 120 a 1 and 120 a 2 of themanagement terminal device CL, respectively. Each of the relay units 120a 1 and 120 a 2 of the management terminal device CL transmits thecombined video image BC acquired from the video conference multipointconnection unit 110A of the management terminal device CL to the Webconference connection units 130 a 1 and 130 a 2. On the other hand, atthe site b1 of the Web conference system MS2, the video image b1 of theconference participant at the site b1 is imaged, acquired in the Webconference terminal 130 b 1, and transferred to the Web conferenceconnection unit 130 a 1 at the virtual site a1. Similarly, at the siteb2 of the Web conference system MS2, the video image b2 of theconference participant at the site b2 is imaged, acquired in the Webconference terminal 130 b 2, and transferred to the Web conferenceconnection unit 130 a 2 of the virtual site a2.

In a case where the video conference system MS11 and the Web conferencesystem MS12 are connected and made to cooperate with each other by therelay units 120 a 1 and 120 a 2 of the management terminal device CL,respectively, each of the relay units 120 a 1 and 120 a 2 of themanagement terminal device CL transmits the combined video image of thevideo conference system MS11 to each of the Web conference connectionunits 130 a 1 and 130 a 2. Each of the Web conference connection units130 a 1 and 130 a 2 of the management terminal device CL inputs thecombined video image BC transferred from the relay units 120 a 1 and 120a 2 of the management terminal device CL, and transmits the combinedvideo image BC to each of the Web conference terminals 130 b 1 and 130 b2 of the other sites b1 and b2. Further, each of the Web conferenceterminals 130 b 1 and 130 b 2 transmits the video images b1 and b2 ofthe own sites b1 and b2 to the Web conference connection units 130 a 1and 130 a 2 of the virtual sites a1 and a2, respectively, and combinesthe video images b1 and b2 of the own sites b1 and b2, and the combinedvideo image BC transferred from each of the other sites a1 and a2 to bedisplayed on the display unit.

Here, in the second embodiment, when each of the relay units 120 a 1 and120 a 2 of the management terminal device CL transmits from the videoconference multipoint connection unit 110A to the Web conferenceconnection units 130 a 1 and 130 a 2 on the Web conference system side,as the first video image data, only the combined video image BC of eachof the sites B and C of the video conference system is transferred.Then, each of the relay units 120 a 1 and 120 a 2 of the managementterminal device CL transfers the combined video image BC from the Webconference connection units 130 a 1 and 130 a 2 to each of the Webconference terminals 130 b 1 and 130 b 2 of the other sites b1 and b2 inthe Web conference system MS12.

Further, when transmitting from the Web conference connection units 130a 1 and 130 a 2 of the management terminal device CL to the videoconference multipoint connection unit 110A on the video conferencesystem MS11 side, each of the relay units 120 a 1 and 120 a 2 of themanagement terminal device CL transfers the video image sent from eachof the Web conference connection units 130 a 1 and 130 a 2 istransmitted, as the second video image data. Then, each of the relayunits 120 a 1 and 120 a 2 of the management terminal device CL transfersthe video image to the video conference terminals 110B and 110C of thesites B and C at the video conference multipoint connection unit 110A ofthe management terminal device CL.

That is, in the first example illustrated in FIG. 13A as one of thespecific examples, at the virtual site A, the video images B and C ofthe own sites from the sites B and C are combined to generate thecombined video image BC, and a combined video image HD-B/C/PC-b1/b2 isfurther generated in which each of the video images b1 and b2 from thesites b1 and b2 is also combined with respect to the combined videoimage BC. The combined video image HD-B/C/PC-b1/b2 is transferred anddisplayed to the video conference terminals 110B and 110C of the sites Band C, respectively. Further, a combined video image HD-B/C/PC-b2 inwhich the video image b2 from the site b2 is combined with the combinedvideo image BC is also further generated at the virtual site A. Thecombined video image HD-B/C/PC-b2 is transferred and displayed to theWeb conference terminal 130 b 1 of the site b1 through the virtual sitesA and a1. Further, the combined video image HD-B/C/PC-b1 in which thevideo image b1 from the site b1 is combined with the combined videoimage BC is also further generated at the virtual site A. The combinedvideo image HD-B/C/PC-b1 is transferred and displayed to the Webconference terminal 130 b 1 of the site b2 through the virtual sites Aand a2.

Then, in the first example illustrated in FIG. 13A, at the site b1, thecombined video image HD-B/C/PC-b2 from the virtual site a1 is disposedin the right half of the display screen of the Web conference terminal130 b 2, and the video image b1 at the own site of the site b1 iscombined and displayed in a state where of being disposed in theremaining left half. On the other hand, at the site b2, the combinedvideo image HD-B/C/PC-b1 from the virtual site a2 is disposed in theright half of the display screen of the Web conference terminal 130 b 2,and the video image b2 at the own site of the site b2 is combined anddisplayed in the state of being disposed in the remaining left half.

Further, as another specific example, in the second example illustratedin FIG. 13B, at the virtual site A, the video images B, b1, and b2 fromthe respective sites from the sites B, b1, and b2 are combined togenerate the combined video image HD-B/PC-b1/b2. The combined videoimage HD-B/PC-b1/b2 is transferred to the video conference terminal 110Cat the site C. On the display screen of the video conference terminal110C of the site C, the combined video image HD-B/PC-b1/b2 is disposedin the right half thereof, and the video image C at the own site of thesite C is disposed and displayed in the remaining left half. Further,the video images C, b1, and b2 from respective sites from the site C,the site b1, and the site b2 are combined to further generate thecombined video image HD-C/PC-b1/b2. The combined video imageHD-C/PC-b1/b2 is transferred to the video conference terminal 110B ofthe site B. On the display screen of the video conference terminal 110Bof the site B, the combined video image HD-C/PC-b1/b2 is disposed in theright half thereof, and the video image B at the own site of the site Bis disposed and displayed in the remaining left half.

Further, at the virtual site A, the video images B, C, and b2 fromrespective sites from the sites B, C, and b2 are combined to generatethe combined video image HD-B/C/PC-b2. The combined video imageHD-B/C/PC-b2 is transferred to the Web conference terminal 130 b 1 ofthe site b1 through the virtual site a1. Further, the video images B, C,and b1 from respective sites from the sites B, C, and b1 are combined togenerate the combined video image HD-B/C/PC-b1. The combined video imageHD-B/C/PC-b1 is transferred to the Web conference terminal 130 b 2 ofthe site b2 through the virtual site a2.

Then, also in the second example illustrated in FIG. 13B, similarly tothe first example described above, at the site b1, the combined videoimage HD-B/C/PC-b2 from the virtual site a1 is disposed in the righthalf of the display screen of the Web conference terminal 130 b 1, andthe video image b1 at the own site of the site b1 is combined anddisplayed in a state of being disposed in the remaining left half. Onthe other hand, at the site b2, the combined video image HD-B/C/PC-b1from the virtual site a2 is disposed in the right half of the displayscreen of the Web conference terminal 130 b 2, and the video image b2 atthe own site of the site b2 is combined and displayed in the state ofbeing disposed in the remaining left half.

As described above, similarly to the first embodiment, the conferencesystem of the second embodiment is operated, so that the same videoimage is not returned and a generation of a phenomenon of a video loop,in which the combined video image is repeatedly transferred, and a videoimage like a mirror is displayed, is prevented.

As described above, in the second embodiment, in the relay units 120 a 1and 120 a 2 of the management terminal device CL, only the combinedvideo image of the video image in at each of the sites B and C of thevideo conference system MS11 is transmitted to the Web conferenceconnection units 130 a 1 and 130 a 2, respectively. The video image sentfrom other sites b1 and b2 of the Web conference system MS12 istransmitted to the video conference multipoint connection unit 110A ofthe management terminal device CL. With this configuration, it ispossible to suppress that the same video image is repeatedly transferredbetween a plurality of conference systems. For that reason, a video loopcan be avoided, an appropriate video image display screen can beobtained at each site, and a smooth video conference can be executed bysharing a conference video image between the sites. Accordingly, it ispossible to improve a display mode when sharing a conference video imageby making a plurality of video conference systems cooperate with eachother.

As described above, the conference system of the first embodimentincludes the video conference system MS11 as an example of the firstconference system and the Web conference system MS12 as an example ofthe second conference system. The video conference multipoint connectionunit 110A as an example of a first transmission/reception unit thattransmits and receives the video image data acquired at the site of thevideo conference system MS11, and the Web conference connection units130 a 1 and 130 a 2 as an example of a second transmission/receptionunit that transmit and receive the video image data acquired at the siteof the Web conference system MS12 are provided. Further, relay units 120a 1 and 120 a 2 are provided which transfers data to each other betweenthe video conference multipoint connection unit 110A and the Webconference connection units 130 a 1 and 130 a 2, transmits the firstvideo image data to the Web conference connection units 130 a 1 and 130a 2, respectively, and transmits the second video image data to thevideo conference multipoint connection unit 110A. The relay units 120 a1 and 120 a 2 transmit only the combined video image of the video imageat each site of the video conference system MS11 as the first videoimage data to the Web conference connection units 130 a 1 and 130 a 2,respectively, and transmit the video image sent from each of the Webconference connection units 130 a 1 and 130 a 2 as the second videoimage data to the video conference multipoint connection unit 110A. Withthis configuration, a video loop is avoided without the same video imagebeing transferred repeatedly between a plurality of conference systems.

In the second embodiment, the video conference multipoint connectionunit 110A transmits the combined video image obtained by combining videoimages at a plurality of sites of the video conference system MS11 toeach of the video conference terminals 110B and 110C as an example ofthe third terminals provided at the respective sites of the videoconference system MS11, and transmits the second video image data sentfrom the relay units 120 a 1 and 120 a 2 to each of the video conferenceterminals 110B and 110C at the respective sites of the video conferencesystem MS11 when transferring data to each of the Web conferenceconnection units 130 a 1 and 130 a 2 by the relay units 120 a 1 and 120a 2. With this configuration, the same video image is not repeatedlytransferred, and the combined video image is generated in a state wherethe video image of each site does not overlap and is displayed on eachterminal. For that reason, it is possible to prevent the occurrence ofthe video loop phenomenon in which the video image like a mirror withrepeated nesting of the combined video image is displayed.

In the second embodiment, each of the Web conference connection units130 a 1 and 130 a 2 transmits the video image at its own site of the Webconference system MS12 to each of the Web conference terminals 130 b 1and 130 b 2 as an example of a fourth terminal provided at other sitesof the Web conference system MS12, and transmits the video image of thefirst video image data sent from the relay units 120 a 1 and 120 a 2 toeach of the Web conference terminals 130 b 1 and 130 b 2 of other sitesof the Web conference system MS12 when transferring data to the videoconference multipoint connection unit 110A by the relay units 120 a 1and 120 a 2. With this configuration, the same video image is notrepeatedly transferred, and the combined video image is generated in astate where the video image of each site does not overlap and isdisplayed on each terminal. For that reason, it is possible to preventthe occurrence of the video loop phenomenon in which the video imagelike a mirror with repeated nesting of the combined video image isdisplayed.

The video conference device according to the second embodiment includesthe relay units 120 a 1 and 120 a 2 that transmit the video image dataacquired at the site of the first conference system (video conferencesystem MS11) to each of the Web conference connection units 130 a 1 and130 a 2 as an example of a second transmission/reception unit providedin the second conference system (Web conference system MS12), andtransmits the video image data of the second conference system to thevideo conference multipoint connection unit 110A as an example of afirst transmission/reception unit provided in the first conferencesystem. In the second embodiment, the video conference device isconfigured as the management terminal device CL including the functionsof the video conference multipoint connection unit 110A, the relay units120 a 1 and 120 a 2, and the Web conference connection units 130 a 1 and130 a 2. Each of the relay units 120 a 1 and 120 a 2 transmits only thecombined video image of the video image at each site of the videoconference system MS11 as the first video image data to each of the Webconference connection units 130 a 1 and 130 a 2, and transmits the videoimage sent from each of the Web conference connection units 130 a 1 and130 a 2 as the second video image data to the video conferencemultipoint connection unit 110A. With this configuration, a video loopis avoided without the same video image being transferred repeatedlybetween a plurality of conference systems.

Modified Example of Second Embodiment

Next, a modified example of the second embodiment will be described. Inthe present modified example, the management terminal device CL is notprovided with the relay unit (120 a 1 and 120 a 2, see FIGS. 11, 12A,and 12B). Instead, in the present modified example, the functions of therelay units (120 a 1 and 120 a 2) are realized by the video conferencemultipoint connection unit 110A of the management terminal device CL,that is, the video conference multipoint connection unit 110A of themanagement terminal device CL of the present modified example alsoincludes a function as the relay units (120 a 1 and 120 a 2).

Specifically, in the present modified example, the video conferencemultipoint connection unit 110A of the management terminal device CL asan example of the first transmission/reception unit transmits data eachother between the Web conference connection units 130 a 1 and 130 a 2 asan example of the second transmission/reception unit, transmits thefirst video image data to each of the Web conference connection units130 a 1 and 130 a 2, and receives the second video image data from eachof the Web conference connection units 130 a 1 and 130 a 2. At the sametime, the video conference multipoint connection unit 110A of themanagement terminal device CL transmits only the combined video image ofthe video image at each site of the video conference system MS11 as thefirst video image data to each of the Web conference connection units130 a 1 and 130 a 2.

In addition, the video conference multipoint connection unit 110Atransmits the combined video image obtained by combining video images ata plurality of sites of the video conference system MS11 to each of thevideo conference terminals 110B and 110C as an example of the thirdterminal provided at the respective sites of the video conference systemMS11, and transmits the second video image data to each of the videoconference terminals 110B and 110C at the respective sites of the videoconference system MS11 when transferring data to each of the Webconference connection units 130 a 1 and 130 a 2.

Further, each of the Web conference connection units 130 a 1 and 130 a 2of the present modified example transmits the video image at its ownsite of the Web conference system MS12 to each of the Web conferenceterminals 130 b 1 and 130 b 2 as an example of the fourth terminalprovided at other sites of the Web conference system MS12, and transmitsthe video image of the first video image data to each of the Webconference terminals 130 b 1 and 130 b 2 of the other sites of the Webconference system MS12 when transferring data to the video conferencemultipoint connection unit 110A. The other configurations and theireffects are the same as those in the second embodiment described above.

Although various embodiments have been described with reference to thedrawings, it goes without saying that the present disclosure is notlimited to such examples. It is obvious to those skilled in the art thatvarious changes or modifications can be made within the scope describedin the claims, and it is understood that those various changes ormodifications naturally belong to the technical scope of the presentdisclosure. Further, constitutional elements in the embodiment describedabove may be combined as occasion demands, without departing from thespirit of the invention.

In addition, the present disclosure may also be applied to a programwhich is for realizing the functions of the video image processingmethod according to the above-described embodiment, is supplied to aninformation processing device (terminal) which is a computer via anetwork or various storage media, and is read and executed by aprocessor of the information processing device, and a recording mediumon which the program is stored.

The present disclosure is useful as a conference system, a videoconference apparatus, and a video image processing method that suppressoccurrence of looping of a video image when transferring a video imageby making a plurality of conference systems cooperate with each other,and allows an appropriate conference video image to be shared betweenthe plurality of conference systems.

What is claimed is:
 1. A conference system comprising: a first terminalthat is communicatively coupled to at least one first additionalterminal, the first terminal being configured to generate first videoimage data and the first terminal being constituted of a server computerfor the cloud; and a second terminal that is communicatively coupled tothe first terminal and at least one second additional terminal, thesecond terminal being configured to generate second video image data andthe second terminal being constituted of a server computer for thecloud, wherein the first terminal is configured to transmit the firstvideo image data to the second terminal, and the second terminal isconfigured to transmit the second video image data to the firstterminal, in a case where a loop avoidance function is disabled, thefirst video image data includes a combined video image that includesvideo images generated by the first terminal, the at least one firstadditional terminal communicatively coupled to the first terminal, thesecond terminal, and the at least one second additional terminalcommunicatively coupled to the second terminal, and in a case where theloop avoidance function is enabled, the first video image data includesa combined video image that includes video images generated by the firstterminal and the at least one first additional terminal communicativelycoupled to the first terminal, and omits video images generated by thesecond terminal and the at least one second additional terminalcommunicatively coupled to the second terminal.
 2. The conference systemaccording to claim 1, wherein the at least one first additional terminalincludes a third terminal and a fourth terminal, and the first terminalis configured to generate and transmit third video image data to thethird terminal, the third video image data including a combined videoimage that includes video images generated by the first terminal and thefourth terminal.
 3. The conference system according to claim 1, whereinthe second terminal is configured to generate and transmit third videoimage data to the at least one second additional terminalcommunicatively coupled to the second terminal, the third video imagedata including the video image generated by the second terminal.
 4. Avideo conference apparatus comprising: a first terminal that iscommunicatively coupled to at least one first additional terminal, thefirst terminal being constituted of a server computer for the cloud andthe first terminal being configured to generate first video image dataand transmit the first video image data to a second terminal, whereinthe second terminal is communicatively coupled to the first terminal andat least one second additional terminal, and the second terminal isconstituted of a server computer for the cloud, the second terminal isconfigured to generate second video image data and transmit the secondvideo image data to the first terminal, in a case where a loop avoidancefunction is disabled, the first video image data includes a combinedvideo image that includes video images generated by the first terminal,the at least one first additional terminal communicatively coupled tothe first terminal, the second terminal, and the at least one secondadditional terminal communicatively coupled to the second terminal, andin a case where the loop avoidance function is enabled, the first videoimage data includes a combined video image that includes video imagesgenerated by the first terminal and the at least one first additionalterminal communicatively coupled to the first terminal, and omits videoimages generated by the second terminal and the at least one secondadditional terminal communicatively coupled to the second terminal.
 5. Avideo image processing method in a conference system, the video imageprocessing method comprising: generating, by a first terminal, firstvideo image data, the first terminal being constituted of a servercomputer for the cloud and the first terminal being communicativelycoupled to at least one first additional terminal; generating, by asecond terminal, second video image data, the second terminal beingconstituted of a server computer for the cloud and the second terminalcommunicatively coupled to the first terminal and at least one secondadditional terminal; transmitting, by the first terminal, the firstvideo image data to the second terminal; transmitting, by the secondterminal, the second video image data to the first terminal, wherein ina case where a loop avoidance function is disabled, the first videoimage data includes a combined video image that includes video imagesgenerated by the first terminal, the at least one first additionalterminal communicatively coupled to the first terminal, the secondterminal, and the at least one second additional terminalcommunicatively coupled to the second terminal, and in a case where theloop avoidance function is enabled, the first video image data includesa combined video image that includes video images generated by the firstterminal and the at least one first additional terminal communicativelycoupled to the first terminal, and omits video images generated by tothe second terminal and the at least one second additional terminalcommunicatively coupled to the second terminal.
 6. The conference systemaccording to claim 1, wherein, in the case where the loop avoidancefunction is enabled, the second video image data includes a combinedvideo image that includes video images generated by the first terminal,the at least one first additional terminal communicatively coupled tothe first terminal, and the at least one second additional terminalcommunicatively coupled to the second terminal.
 7. The conference systemaccording to claim 1, wherein the second video image data includes videoimages generated by the second terminal and the at least one secondadditional terminal communicatively coupled to the second terminal. 8.The conference system according to claim 2, wherein the second videoimage data includes video images generated by the second terminal andthe at least one second additional terminal communicatively coupled tothe second terminal, and the third video image data includes a combinedvideo image that includes the video images generated by the firstterminal and the fourth terminal and the video images generated by thesecond terminal and the at least one second additional terminalcommunicatively coupled to the second terminal.
 9. The conference systemaccording to claim 1, wherein in the case where the loop avoidancefunction is enabled, the second terminal is configured combine anddisplay the video image generated by the at least one second additionalterminal communicatively coupled to the second terminal and the combinedvideo image included in the first video image data.
 10. The conferencesystem according to claim 1, wherein the second terminal is configuredto transmit the first video image data to the at least one secondadditional terminal communicatively coupled to the second terminal. 11.The conference system according to claim 1, wherein the first terminalis a server computer for the cloud different from a server computer forthe cloud of the second terminal.
 12. The conference system according toclaim 1, wherein a conference system of the first terminal is aconference system different from a conference system of the secondterminal.
 13. The conference system according to claim 1, wherein thefirst terminal is a video conference system and the second terminal is aWeb conference system.
 14. The conference system according to claim 1,wherein the first terminal has a different communication protocol fromthat of the second terminal.
 15. The conference system according toclaim 1, wherein the first terminal is configured to transmit the firstvideo image data to the second terminal via a relay unit, and the secondterminal is configured to transmit the second video image data to thefirst terminal via the relay unit.
 16. The conference system accordingto claim 1, wherein the first terminal and the second terminal areconnected by software, the first terminal and the first additionalterminal are connected by a communication interface, and the secondterminal and the second additional terminal are connected by acommunication interface.